A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization

We introduce a tunable loss function called $\alpha $ -loss, parameterized by \in (0,\infty]$ , which interpolates between the exponential ( = 1/2$ ), log-loss 1$ and 0–1 \infty for machine learning setting of classification. Theoretically, we illustrate fundamental connection -loss Arimoto conditional entropy, verify classification-calibration in order to demonstrate asymptotic optimality via Rademacher complexity generalization techniques, build-upon notion strictly local quasi-convexity quantitatively characterize optimization landscape -loss. Practically, perform class imbalance, robustness, classification experiments on benchmark image datasets using convolutional-neural-networks. Our main practical conclusion is that certain tasks may benefit from tuning away this end provide simple heuristics practitioner. In particular, navigating hyperparameter can readily superior model robustness label flips > ) sensitivity imbalanced classes < ).